1. Field of the Invention
The present invention relates generally to communications over a network, and specifically, to establishing connections between clients and servers.
2. Description of Background Art
In the development of a large computing services site supporting an Internet, LAN or WAN based service, such as an e-commerce site or a telecommunications service site, scalability may be achieved by employing multiple servers to service a plurality of clients in a network. In large sites, a load balancer may receive client requests for connections and then redirect the requests to one of the servers that is best able to handle the request at the time the request is received. The clients for such a site may include, for example, a web browser or other application. The services provided may be e-mail services, delivery of web content or any other service which requires establishing a connection between the client and the server.
Establishing a new client connection to a server requires allocation of resources associated with that server. Each server supports the establishment of a finite number of new client connections per second. If this rate is exceeded, the server will become overloaded, and its performance will degrade, resulting in increased latency when establishing client connections, slow response times for established connections and/or the dropping of established client connections.
To eliminate overloading a server with new connections, conventional load balancers employ a technique known as connection pacing. When a new client connection request is directed to a server, using connection pacing technique, subsequent client requests are directed to other servers or are dropped until a specified period of time has passed (a “server pacing period”). Connection pacers, which implement connection pacing, must determine when to remove a server placed into a pacing state. Some connection pacers also determine when to put a server into a pacing state.
Conventional implementations of connection pacers have used a crawler approach to determine when a server may be removed from a pacing state. In such an approach, entries are maintained in a database, with one entry for each server, each entry containing information about the time the server may be removed from pacing state. When a connection is established, the time the server is next available (time_next_avail) is written into the crawler database. A separate crawler continually and sequentially trolls through all entries in the database and checks time_next_avail against the current time. When time_next_avail is less than the current time, the server has been in pacing state for the required server pacing period and is ready to be removed from such pacing state. To remove a server from a pacing state, a modification is made to the entry in the database associated with that server to change the pacing state of that server. Implementing a connection pacer using the technique described in this paragraph above results in:
(a) An increase in time in locating and removing from a pacing state a server whose server pacing period has expired;
(b) A reduction in the rate at which new connections can be established;
(c) A reduction in the number of clients that can be supported due to sub-optimal use of server resources;
(d) A reduction in accuracy in the time between establishing a connection to a server and subsequently making that server available for new connections; and
(e) An increase in the minimum time between establishing a connection to a server and subsequently making that server available for new connections.